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Abstract: Computer-assisted interviewing techniques have increasingly 
been used in program and research settings to improve data collection 
quality and efficiency. Little is known, however, regarding the use of such 
techniques with American Indian (AI) adolescents in collecting sensitive 
information. This brief compares the consistency of AI adolescent mothers ’ 
reporting of sensitive sexual and drug use behaviors gathered through 
three distinct interviewing techniques: computer-assisted (ACASI), self- 
administered questionnaire (SAQ), and face-to-face interview (FTFI). 
Endorsement of drug use and reporting of sexual activity was highest 
for ACASI, followed by SAQ, and was significantly lower for FTFI. 
Relatively strong agreement was measured between ACASI and SAQ, and 
relatively poor agreement was measured between the ACASI and FTFI. 
Findings support the use of computer-assisted interviewing techniques 
with AI adolescents, and implications for future research are discussed. 


INTRODUCTION 

Computer-assisted interviewing techniques have been increasingly explored in the last decade 
to improve reporting of sensitive behaviors. Compared to self-report or interview data collected on 
paper, audio computer-assisted interviewing (ACASI) can overcome literacy constraints by allowing 
participants to listen to questions over headphones, improve data quality and efficiency as data are 
uploaded directly as participants answer questions, and improve data validity by affording respondents 
greater privacy in answering questions. Available evidence suggests ACASI techniques yield higher 
reporting of both sexual behaviors (Langhaug, Sherr, & Cowan, 2010; Phillips, Gomez, Body, & 
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Garnett, 2010) and drug use (Turner et al., 1998) compared to paper-and-pen self-administered 
questionnaires (SAQ) and in-person face-to-face interviews (FTFI)—although some cross-cultural 
differences have been found (Jaya, Hindin, & Ahmed, 2008). 

Computer-assisted interviewing has rarely been used in research with American Indian (AI) 
communities. Low use is commonly attributed to resource constraints and concerns regarding 
computer literacy (Edwards et al., 2007). However, when ACASI was used to gather dietary and 
lifestyle information among Southwestern AIs, authors reported high usability and acceptance, 
particularly among younger participants (ages 18-49 years; Edwards et al., 2007; Murtaugh et 
al., 2010). Similarly, Redwood et al. (2010) successfully utilized ACASI techniques to examine 
tobacco use among AIs ages > 18 years. To our knowledge, there are no previous studies comparing 
ACASI to other interviewing techniques among AI adolescents. The availability of accurate and 
valid data on sexual and drug use behaviors is critical for AI reservation communities due to urgent 
behavioral health disparities (Whitesell et al., 2007; Eaton et al., 2012) and the need for evidence- 
based intervention. 

This brief compares consistency of data on sensitive sexual and drug use behaviors gathered 
through three interviewing techniques—ACASI, SAQ, and FTFI—as part of a randomized controlled 
trial in four Southwestern reservations among adolescent AI mothers. 

METHODS 

A total of 322 AI expectant adolescent mothers were enrolled during pregnancy in the 
Family Spirit Trial. The research goal was to evaluate the impact of a paraprofessional-delivered 
home- visiting intervention to reduce health and behavioral risks for AI teen mothers and children. 
The trial was conducted from 2006-2011, and its methods and participant characteristics have 
been previously published (Mullany et al., 2012). All study methods were approved by a total of 
11 tribal research review and advisory boards and tribal councils, the Phoenix Area Indian Health 
Service Institutional Review Board, and the Johns Hopkins Bloomberg School of Public Health 
Institutional Review Board. The intervention and its evaluation spanned a period of 39 months, 
from participants’ third trimesters through their children’s third birthdays. At the children’s second 
birthdays, a methodological substudy was embedded in the trial to compare data quality on sensitive 
behaviors as gathered through AC ASI, SAQ, and FTFI. 

All three interviewing techniques (ACASI, SAQ, and FTFI) were administered to participants 
in a private location (home, vehicle, or private study office). Responses to drug use questions were 
compared across ACASI, SAQ, and FTFI formats. Sexual behavior responses were compared only 
between ACASI and FTFI because these data were not gathered via SAQ. 


American Indian and Alaska Native Mental Health Research 
Copyright: Centers for American Indian and Alaska Native Health 
Colorado School of Public Health/University of Colorado Anschutz Medical Campus ( www.ucdenver.edu/caianh) 




44 VOLUME 20, NUMBER 2 


Drug use and sexual behavior questions were adapted from instruments previously used 
with AI populations, including the Voices of Indian Teens Project (Novins & Mitchell, 1998) and 
the Montana Meth Survey (Montana Meth Project, 2008). Drug use questions were asked nearly 
identically across all three formats (ACASI, SAQ, and FTFI). For example, lifetime use of marijuana 
was ascertained by asking “Have you ever tried marijuana?” on both the SAQ and ACASI. Past 
month use of marijuana was assessed by asking “On how many days did you use marijuana in the 
past month?” (ACASI), “How many times did you use marijuana in the last month?” (SAQ), and 
“How many times did you use marijuana in the past 30 days?” (FTFI). Sexual behavior questions 
aimed to gather the same data but used slightly different wording across ACASI and FTFI formats. 
Specifically, current sexual activity was asked as “Are you currently having sex?” on ACASI and 
as “Are you currently sexually active?” on FTFI. Contraceptive use was assessed by asking “Are 
you currently doing anything to prevent getting pregnant?” on ACASI and by asking “Are you 
currently using any method of protection?” on FTFI. 

Analyses were limited to participants who completed different interview formats within a 
one-week period (n = 92 for women with ACASI, SAQ, and FTFI, n = 129 for women with SAQ 
and ACASI, and n = 110 for women with SAQ and FTFI). Since these analytic sample sizes well 
exceed the minimum number of approximately 66 individuals needed in interrater agreement studies 
to detect statistically significant kappa values at the p < 0.05 level for dichotomous variables (Sim 
& Wright, 2005), we had at least 90% power for each of our statistical tests. 

We present risk behaviors by mode of interview using percentages. We calculated ratios of 
affirmative responses and their confidence intervals, and tested significance using McNemar’s chi- 
squared statistic for matched case-control data. We calculated Cohen’s kappa coefficients to measure 
agreement in responses to the same questions across different interview formats. Kappa values 
are interpreted as follows: 0.0-0.2 = poor agreement, 0.2-0.4 = fair agreement, 0.4-0.6 = moderate 
agreement, 0.6-0.8 = good agreement, and 0.8-1.0 = very good agreement. Mean differences in 
quantitative data were calculated using paired t- tests. 

RESULTS 

Respondents were ages 15-23 years (mean = 20.2). Unfortunately, our sample was not large 
enough to stratify by age groups with statistical significance, so we did not examine age differences 
between younger and older adolescents. Reported drug use and sexual behaviors are compared in 
Tables 1 and 2. 
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Table 1 

Reported Endorsement of Past Month and Lifetime use of Drugs and Current Sexual 

Behaviors Across Interview Formats 3 


Interview Format 


ACASI 

SAQ 

FTFI 

Endorsement of Behavior 

(% yes) 

(% yes) (% yes) 

For women with ACASI, SAQ, and FTFI completed within one week of each other (n = 

92) 

Past month alcohol use 

16.3% 

13.0% 

8.7% 

Past month marijuana use 

14.1% 

13.0% 

2.2% 

Past month crack/cocaine use 

2.2% 

2.2% 

0.0% 

Past month methamphetamine use 

2.2% 

3.3% 

1.1% 

For women with ACASI and SAQ completed within one week of each other {n = 129) 


Ever used alcohol 

69.8% 

73.4% 

N/A 

Ever used marijuana 

65.9% 

61.2% 

N/A 

Ever used crack/cocaine 

15.5% 

12.4% 

N/A 

Ever used methamphetamine 

24.0% 

14.7% 

N/A 

For women with ACASI and FTFI completed within one week of each other (n = 110) 


Currently sexually active 

51.8% 

N/A 

35.5% 

Currently using any contraception 

30.0% 

N/A 

22.7% 

Method of contraception: 




Pill 

0.9% 

N/A 

2.7% 

Condom 

15.5% 

N/A 

16.4% 

Depo-Provera 

10.0% 

N/A 

2.7% 

Withdrawl 

0.9% 

N/A 

0.0% 

Rhythm 

5.5% 

N/A 

0.0% 


a ACASI = audio computer-assisted interview, SAQ = self-administered questionnaire, FTFI = face- 
to-face interview 
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Table 2 

Comparison and Agreement Test Results of Reported Past Month and Lifetime Use of Drugs and 
Current Sexual Behaviors Across Interview Formats 3 

ACASI versus SAQ 

Ratio of . % Said % Said 

Affirmative _ a IO Kappa Yes on SAQ Yes on ACASI 
Responses on . Score but but 

Endorsement of Behavior ACASI vs. SAQ n erVa No on ACASI No on SAQ 

For women with ACASI, SAQ and FTFI completed within one week of each other (n = 92) 


Past month alcohol use 

1.25 

0.85-1.84 

0.70 

2.2% 

5.4% 

Past month marijuana use 

1.08 

0.72-1.64 

0.68 

3.3% 

4.3% 

Past month crack/cocaine use 

1.00 

1 . 00 - 1.00 

1.00 

0.0% 

0.0% 

Past month methamphetamine use 

0.67 

0.17-2.67 

0.38 

2.2% 

1.1% 

For women with ACASI and SAQ completed within one week of each other (n 

= 129) 


Ever used alcohol 

0.96 

0.88-1.04 

0.69 

7.8% 

4.6% 

Ever used marijuana 

1.06 

0.97-1.16 

0.78 

3.1% 

7.0% 

Ever used crack/cocaine 

1.19 

0.92-1.53 

0.83 

0.8% 

3.1% 

Ever used methamphetamine 

1.63** 

1.23-2.16 

0.71 

0.0% 

9.3% 

ACASI versus FTFI 

Endorsement of Behavior 

Ratio of 
Affirmative 
Responses on 
ACASI vs. FTFI 

Ratio 

Confidence 

Interval 

Kappa 

Score 

% Said 

Yes on FTFI 
but 

No on ACASI 

% Said 

Yes on ACASI 
but 

No on FTFI 

For women with ACASI, SAQ and FTFI completed within one week of each other (n = 92) 


Past month alcohol use 

1.88* * 

1.10-3.21 

0.56 

1.1% 

8.7% 

Past month marijuana use 

6.50** 

1.82-23.26 

0.24 

0.0% 

12.0% 

Past month crack/cocaine use 

- 

- 

0.00 

0.0% 

2.2% 

Past month methamphetamine use 

2.00 

0.50-7.99 

0.66 

0.0% 

1.1% 

For women with FTFI and ACASI completed within one week of each other (n 

= 110) 


Currently sexually active 

1.46** 

1.18-1.81 

0.53 

3.6% 

20.0% 

Currently using any contraception 
Method of contraception: 

1.32 

0.97-1.79 

0.53 

5.4% 

12.7% 

Pill 

0.33 

0.03-3.20 

-0.01 

2.7% 

0.9% 

Condom 

0.94 

0.65-1.37 

0.63 

5.4% 

4.5% 

Depo-Provera 

3.67** 

1.40-9.62 

0.40 

0.0% 

7.3% 

Withdrawl 

- 

- 

0.00 

0.0% 

0.9% 

Rhythm 

- 

- 

0.00 

0.0% 

5.4% 


a ACASI = audio computer-assisted interview, SAQ = self-administered questionnaire, FTFI = face-to-face 

*p < .05, **p < .01 for McNemar's significance probabilities from matched case-control analyses 
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Proportions reporting past month drug use were similar, and in some cases slightly higher, 
for ACASI versus SAQ (e.g., 16% vs. 13% for alcohol, 14% vs. 13% for marijuana). Past month 
alcohol (ratio = 1.88,/? < 0.05) and marijuana (ratio = 6.50,/? < 0.01) use were significantly higher on 
ACASI versus FTFI. Kappa values comparing ACASI and SAQ formats indicated good agreement 
for past month use of alcohol (0.70), marijuana (0.68), and cocaine (1.00), and fair agreement for 
methamphetamine use (0.38). Lower overall agreement was found between ACASI and FTFI (kappas 
ranging from 0-0.66). 

With the exception of alcohol, reported lifetime drug use was higher on ACASI versus SAQ, 
including: marijuana (66% vs. 61%), cocaine (16% vs. 12%) and methamphetamine (24% vs. 15%). 
For methamphetamine, there was significant difference in ACASI versus SAQ formats (ratio = 1.63, 
p < 0.01). Agreement levels for lifetime use between ACASI and SAQ were in the good to very 
good range: methamphetamine (kappa = 0.71), marijuana (0.78), cocaine (0.83), and alcohol (0.69). 

Regarding age at initiation, participants on average first used marijuana (mean age = 14.3 
years, SD = 2.1), followed by alcohol (mean age = 14.6 years, SD = 2.2 years), crack/cocaine 
(mean age = 15.4 years, SD = 1.8), and methamphetamine (mean age = 16.6 years, SD = 1.8). No 
significant differences were found for mean age of first drug use for ACASI versus SAQ (unadjusted 
mean differences ranged from -0.22 to 0.53 and p values for t-test results ranged from 0.31-0.94; 
data not shown). 

For sexual behaviors, a significantly larger proportion of participants reported being sexually 
active (52% vs. 36%) on ACASI versus FTFI (ratio = 1.46,/? < 0.01). Though not statistically 
significant, more women reported using contraception on ACASI versus FTFI (30% vs. 23%). 
Agreement levels were in the moderate range (kappa = 0.53 for both). Agreement for reported 
contraception was highest for condom use (kappa = 0.63) and lowest for birth control pills, withdrawal, 
rhythm, and Depo-Provera (kappa values ranging from -.01 to 0). The only birth control with 
significant differences in reporting was Depo-Provera, with 3.67 times more women endorsing it 
on ACASI {p < 0.01). 


DISCUSSION 

Endorsement of drug use behaviors among this sample of AI adolescent mothers was 
highest for ACASI, followed by SAQ, and was significantly lower for FTFI. These patterns were 
seen consistently across different drug use questions, despite the similar contexts of question 
administration (e.g., question wording, setting and timing of data collection,). Kappa values indicated 
overall good to very good agreement between ACASI and SAQ, and poor to moderate agreement 
between the ACASI and FTFI. Quantitative questions (i.e., age of first use) were less affected by 
interview format. 
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The largest differentials in drug use reporting across data collection formats were seen for 
lifetime use of methamphetamine and past month use of alcohol and marijuana, with ACASI eliciting 
higher rates of reported use than SAQ or FTFI. While we were unable to determine what the “truth” 
actually was for these participants, we believe ACASI is likely to yield the most valid data, because 
participant response bias is more likely to lead youth to under-report, rather than over-report, illicit 
behavior. For example, in reporting of lifetime methamphetamine use, all cases of disagreement 
were attributed to respondents saying yes on ACASI and no on SAQ. 

We feel it is easier to identify and understand possible motivations for youth to answer 
incorrectly and report that they had not used drugs when they really had (particularly when a 
study team member is conducting the interview or sitting nearby) than to answer incorrectly on a 
computer-administered questionnaire and say they had used drugs when they really had not. It is 
likely that participants perceived ACASI to be more confidential. 

Our data on reporting of sexual behaviors were limited to comparisons between ACASI 
and FTFI. Respondents were significantly more likely to report sexual behaviors on ACASI versus 
FTFI, even though FTFI data were gathered by a trusted community member and eligibility criteria 
included being pregnant (thus revealing previous sexual activity). Regarding reported methods 
of contraception, agreement levels were highest for condoms and lowest for birth control pills, 
withdrawal, rhythm methods, and Depo-Provera. It is unclear why disagreement was so high for 
Depo-Provera, given it is not self-administered; this finding deserves further exploration. 

Several other factors may have influenced the findings reported here. Agreement levels for 
drug use and sexual behavior reporting may have been influenced by the slightly different question 
wording across assessment formats. In most cases, however, the wording differences were so minor 
(e.g., “in the past month” vs. “in the last month,” “past month” vs. “past 30 days”), that they were 
unlikely to have significantly influenced disagreement levels. The exception is contraceptive use, 
where the differences in question wording were more substantial; it is possible that participants’ 
differing interpretations of the varied wording contributed to the measured disagreement levels in 
contraceptive use reporting. 

Additionally, kappa coefficients are, in part, a function of the prevalence of an attribute (in 
this case, an affirmative response of the sensitive behavior). In situations of very low prevalence, 
chance agreement is higher, thereby reducing the value of the corresponding kappa statistic (Sim & 
Wright, 2005). The relatively low prevalence of past month use of methamphetamine and of crack/ 
cocaine may have contributed to the lower kappa values for these questions. 

To our knowledge, this is the first paper providing results from a methodological examination 
of three interviewing techniques among AI adolescents. This research supports the use of ACASI with 
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AI adolescents, given the urgent need for understanding high-risk behaviors, the utility of ACASI 
to overcome literacy issues, and ACASI’s contribution to efficient data management. 

Future research should examine whether or how age, education levels, or computer literacy 
influence agreement levels; perhaps individuals that are older, have more formal education, or have 
greater computer exposure would be more comfortable with the different interviewing techniques 
and less likely to modify their responses across assessment formats. In addition, the behavioral health 
field would be strengthened by further in-depth examination of the reporting differences found here. 
Conducting follow-up qualitative interviews with youth that have contributed discordant responses 
across data collection formats would help to determine underlying factors that influence reporting 
of sensitive behaviors, which could, in turn, strengthen the ability of behavioral and mental health 
program planners and researchers to implement the most appropriate and valid data collection 
methodologies in their local contexts. 
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